Stat 538 Project: Implementation of Maximum Entropy Discrimination with a Linear Discriminant Function

For this project, our goal was to implement Maximum Entropy Discrimination (MED) with a linear discrimination function for binary classification purposes. This technique was presented by Jaakola, Meila, and Jebara in “Maximum Entropy Discrimination” (1999). Their paper provides a generalized framework for discrimination that relies on the maximum entropy principle. One of the interesting aspects of MED is its consideration of distributions over parameters in the discriminative model. Thus, instead of finding a single optimal setting of paramters, MED estimates a distribution over the parameters and this distribution is chosen to satisfy constraints in a manner that is non-prejudiced or does not imply that possession of information not present. We apply this technique with a linear discriminant function first to a common benchmarking dataset for discrimination techniques and second to a classification problem of interest to the author.